Image pickup system and image processing program

ABSTRACT

An image pickup system includes a color noise estimation section, which has a local region extraction section that extracts luminance and color difference signals in units of 4×4 pixels from signals of a CCD with a color filter, an average luminance calculation section that calculates an average luminance value from the luminance signals, a temperature estimation section that estimates a temperature T of the CCD from a signal of an OB region, a gain calculation section that calculates a gain G of the signals based on information from a control section, a coefficient and a function calculation sections for estimating a color noise amount from the average luminance value based on a formulation using the temperature and the gain, and a color noise reducing section that reduces color noise in the color difference signals in units of 4×4 pixels based on the estimated color noise amount.

CROSS REFERENCE TO RELATED APPLICATION

This application is a continuation application of PCT/JP2004/018027filed on Dec. 3, 2004 and claims benefit of Japanese Application No.2003-410884 filed in Japan on Dec. 9, 2003, the entire contents of whichare incorporated herein by this reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image pickup system and an imageprocessing program, and more particularly to an image pickup system andan image processing program which reduce random noise and the like incolor signals due to an image sensor system.

2. Description of the Related Art

Generally noise components are contained in digitalized signals obtainedfrom an image sensor and its analog circuit and A/D converter, and thesenoise components can be broadly classified into fixed pattern noise andrandom noise.

The fixed pattern noise is typically represented by defective pixels andmainly originates in the image sensor.

On the other hand, the random noise is generated in the image sensor andthe analog circuit and has characteristics similar to thecharacteristics of white noise.

In regard to the latter of these, i.e., random noise, Japanese PatentApplication Laid-open No. 2001-157057 discloses a technique in which anoise amount N is formulated by a function N=ab^(cD) where referencesymbols a, b, and c denote statically given constant terms and thesignal level D is a converted into a density value, the noise amount Nis estimated with respect to the signal level D from this function, andthe filtering frequency characteristics are controlled based on theestimated noise amount N. As a result, adaptively noise reductionprocessing is performed with respect to the signal level.

Also, Japanese Patent Application Laid-open No. 2001-175843 as anotherexample discloses a technique in which the input signals are separatedinto luminance signals and color difference signals, then edge intensityis determined based on these luminance signals and color differencesignals, and a smoothing process is carried out in the color differencesignals for regions other than the edge portions. Thus, color noisereduction processing is carried out in flat regions.

However, since the luminance noise amount varies dynamically accordingto factors such as the temperature at the time of shooting, the exposuretime, the gain and the like, a technique using static constant termssuch as that described in Japanese Patent Application Laid-open No.2001-157057 cannot formulate into a function that matches the noiseamount at the time of shooting and lacks accuracy in estimating thenoise amount. Furthermore, although the filtering frequencycharacteristics are controlled from the noise amount, this filtering iscarried out uniformly without distinguishing between flat regions andedge regions, and therefore the edge deteriorate in regions estimatedthat the noise amount is large from the signal level. That is, there areproblems in that there is no capacity for handling processing thatdistinguishes the original signals and the noise and there is poormaintainability of the original signals. Further still, the techniquedescribed in the aforementioned application has no capacity for handlingcolor noise produced between color signals.

Also, although the technique described in Japanese Patent ApplicationLaid-open No. 2001-175843 carries out a smoothing process on colordifference signals in flat regions except for the edge regions, thissmoothing process is carried out fixedly. However, since the color noiseamounts vary depending on the signal levels, optimal control of thissmoothing process cannot be achieved. Thus, there is a likelihood thatcolor noise components will remain and the original signals willdeteriorate.

The present invention has been devised in light of these circumstancesand it is an object thereof to provide an image pickup system and animage processing program capable of reducing color noise with highaccuracy and optimized to the shooting conditions and producing highquality images.

SUMMARY OF THE INVENTION

An image pickup system of the present invention is an image pickupsystem for reducing noise contained in signals from an image sensor, infront of which is arranged a color filter, and comprises: calculationmeans for calculating luminance signals and color difference signalsfrom the signals in each of predetermined unit regions, calculating anaverage luminance value based on the calculated luminance signals, andcalculating an average color difference value based on the calculatedcolor difference signals; color noise estimation means for estimating acolor noise amount for each predetermined unit region, and color noisereducing means for reducing color noise in the color difference signalsbased on the color noise amount for each predetermined unit region.

Also, an image processing program of the present invention is a programfor having a computer executing: a calculation procedure for calculatingluminance signals and color difference signals for each predeterminedunit region from signals from an image sensor, in front of which isarranged a color filter, calculating an average luminance value based onthe calculated luminance signals, and calculating an average colordifference value based on the calculated color difference signals; acolor noise estimation procedure for estimating a color noise amount foreach predetermined unit region, and a color noise reducing procedure forreducing color noise in the color difference signals based on the colornoise amount for each predetermined unit region.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating a configuration of an imagepickup system according to embodiment 1 of the present invention.

FIG. 2 is a block diagram illustrating a configuration of a color noiseestimation section according to embodiment 1.

FIG. 3 illustrates an arrangement of a Bayer-type color filter accordingto embodiment 1.

FIG. 4 illustrates an arrangement example of an OB region in an imagesensor of embodiment 1.

FIG. 5 is a diagram illustrating a relationship between OB regionvariance and image sensor temperature in embodiment 1.

FIG. 6A and FIG. 6B are a diagram for describing formulation of colornoise amount in embodiment 1.

FIG. 7A and FIG. 7B are a diagram for describing parameters used incalculating color noise amounts in embodiment 1.

FIG. 8 is a block diagram illustrating a configuration of a color noisereducing section in embodiment 1.

FIG. 9 is a flow chart illustrating a color noise reducing processcarried out by an image processing program in a computer according toembodiment 1.

FIG. 10 is a block diagram illustrating a configuration of an imagepickup system according to embodiment 2 of the present invention.

FIG. 11 is a block diagram illustrating an example of a configuration ofa luminance and color noise estimation section according to embodiment2.

FIG. 12A and FIG. 12B illustrate an arrangement of a color-differenceline-sequential-type color filter according to embodiment 2.

FIG. 13 is a block diagram illustrating a configuration of a luminancenoise reducing section according to embodiment 2.

FIG. 14 is a block diagram illustrating another example of aconfiguration of a luminance and color noise estimation sectionaccording to embodiment 2.

FIG. 15 is a flow chart illustrating a noise reducing process carriedout by an image processing program in a computer according to embodiment2.

The following is a description of embodiments of the present inventionwith reference to the accompanying diagrams.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT(S) Embodiment 1

FIGS. 1 to 9 illustrate embodiment 1 of the present invention. FIG. 1 isa block diagram illustrating a configuration of an image pickup system,FIG. 2 is a block diagram illustrating a configuration of a color noiseestimation section, FIG. 3 illustrates an arrangement of a Bayer-typecolor filter, FIG. 4 illustrates an arrangement example of an OB regionin an image sensor, FIG. 5 is a diagram illustrating an example of arelationship between OB region variance and image sensor temperature,FIG. 6A and FIG. 6B are a diagram for describing formulation of colornoise amount, FIG. 7A and FIG. 7B are a diagram for describingparameters used in calculating color noise amounts, FIG. 8 is a blockdiagram illustrating a configuration of a color noise reducing section,and FIG. 9 is a flow chart illustrating a color noise reducing processcarried out by an image processing program in a computer.

As shown in FIG. 1, the image pickup system comprises a lens system 1for forming an image of a subject, an aperture 2 that is arranged insidethe lens system 1 and is for prescribing a range of passage of aluminous flux in the lens system 1, a low-pass filter 3 for eliminatingunnecessary high frequency components from luminous flux which has beenformed into an image by the abovementioned lens system 1, a CCD 4 thatphotoelectrically converts the optical image of a subject formed via thelow-pass filter 3 and outputs an electric image signal, for example, animage sensor such as a single CCD having a primary color filter arrangedin front, a CDS (correlated double sampling) 5 that carries outcorrelated double sampling on the image signal that is output from theCCD 4, an amplifier 6 that amplifies the signal outputted from the CDS5, an A/D converter 7 that converts the analog image signal amplified bythe amplifier 6 to a digital signal, an image buffer 8 that temporarilystores the digital image data that is output from the A/D converter 7,an exposure control section 9 that carries out photometric evaluationrelating to the image of a subject based on the image data stored in theimage buffer 8 and carries out control of the aperture 2, the CCD 4, andthe amplifier 6 based on an evaluation result thereof, a focus controlsection 10 which detects the focal point based on the image data storedin the abovementioned image buffer 8, and which controls an AF motor 11,which is described later, based on the results of detection, an AF motor11 that is controlled by the focus control section 10 and drives acomponent such as a focus lens contained in the lens system 1, apre-white balance (PreWB) section 12 that carries out white balanceadjustment during pre-image capture mode in such ways as varying therate of amplification for each color by the amplifier 6 based on theimage data stored in the image buffer 8, a color noise estimationsection 14 constituting color noise estimation means which performscolor noise estimation in a manner that is described in detail laterbased on image data stored in the image buffer 8, a color noise reducingsection 13, which is color noise reducing means for reducing color noisein the image data read out from the image buffer 8 using an estimationresult of the color noise estimation section 14, a signal processingsection 15 that executes various types of signal processing on the imagedata outputted from the color noise reducing section 13, an outputsection 16 that outputs the image data from the signal processingsection 15 for recording on a memory card for example, an external I/Fsection 17 provided with an interface for such components as a powerswitch, a shutter button, and a mode switch for switching betweenvarious photographing modes, and a control section 18 constituting aparameter calculation means and a control means comprising amicrocomputer or the like which is bidirectionally connected to the CDS5, the amplifier 6, the A/D converter 7, the exposure control section 9,the focus control section 10, the PreWB section 12, the color noisereducing section 13, the color noise estimation section 14, the signalprocessing section 15, the output section 16, and the external I/Fsection 17, so that the control section 18 comprehensively controls theimage pickup system containing these connected sections.

Next, a flow of signals in the image pickup system shown in FIG. 1 isdescribed.

The image pickup system is configured to be able to set shootingconditions such as ISO speed via the external I/F section 17, and afterthese settings have been made, the pre-image-pickup mode is entered byhalf pressing a shutter button formed by a two stage press buttonswitch.

The image signal, which is shot by the CCD 4 via the lens system 1, theaperture 2, and the low-pass filter 3 and then outputted, undergoescommonly known correlated double sampling in the CDS 5 and outputted asan analog signal.

It should be noted that in the present embodiment, the CCD 4 is a singleCCD having a primary color filter arranged in front, and the colorfilter is described using a Bayer-type arrangement as shown in FIG. 3 asan example.

A configuration of a Bayer-type color filter is described with referenceto FIG. 3.

A Bayer type has a 2×2 pixel array as a basic unit with green (G) colorfilters arranged in two pixels diagonally and a red (R) color filter anda blue (B) color filter arranged diagonally in the remaining pixels.

The analog signal from the CDS 5 is amplified by a predetermined amountby the amplifier 6, converted to a digital signal by the A/D converter7, then transferred to the image buffer 8.

After this, the image signals inside the image buffer 8 are transferredto the exposure control section 9, the focus control section 10, and thepre-white balance section 12.

The exposure control section 9 determines the luminance level of theimage, and controls an aperture value of the aperture 2, an electricshutter speed of the CCD 4, an amplification rate of the amplifier 6 andthe like in consideration of the ISO speed and shutter speed of thelimit of image stability, so that an appropriate exposure is obtained.

Furthermore, the focus control section 10 detects edge intensity in theimage and obtains a focused image by controlling the AF motor 11 so asto become maximal edge intensity.

Further, the pre-white balance section 12 calculates simple whitebalance coefficients by multiplying each color signal by a signal with aspecified luminance level in the image signal and transmits thesecoefficients to the amplifier 6. The amplifier 6 carries out whitebalance adjustment by multiplying a different gain for each color signalbased on the coefficient transferred from the pre-white balance section12.

After preparation for the main shooting is in place by carrying out thepre-image-pickup mode, the main shooting is carried out by detecting viathe external I/F section 17 that the shutter button has been fullypushed.

The main shooting is carried out according to the exposure conditionsdetermined by the exposure control section 9, the focus conditionsdetermined by the focus control section 10, and the white balancecoefficient determined by the pre-white balance section 12, and theseconditions of shooting at this time are transferred to the controlsection 18.

When the main shooting is carried out in this way, the image signals aretransferred and stored in the image buffer 8 in the same manner asduring pre-image-pickup.

The image signals in the image buffer 8 are transferred to the colornoise estimation section 14, furthermore, the exposure conditionsdetermined by the exposure control section 9, the white balancecoefficient determined by the pre-white balance section 12, and theshooting conditions such as the ISO speed that has been set using theexternal I/F section 17 are also transferred together to the color noiseestimation section 14 via the control section 18.

Based on this information and the image signals, the color noiseestimation section 14 calculates a color noise amount for eachpredetermined size, for example, in units of 4×4 pixels in the presentembodiment, and transfers the calculated color noise amount to the colornoise reducing section 13. The calculation of color noise amount in thecolor noise estimation section 14 is carried out under the control ofthe control section 18 in synchronization with the processing of thecolor noise reducing section 13.

Based on the color noise amount calculated by the color noise estimationsection 14, the color noise reducing section 13 carries out color noisereduction processing on the image signals in the image buffer 8, andtransfers the processed image signals to the signal processing section15.

Under the control of the control section 18, the signal processingsection 15 carries out commonly known enhancing process, compressionprocess and the like on the image signals that have been subjected tocolor noise reduction processing, and transfers the thus-processed imagesignals to the output section 16.

The output section 16 records and saves the received image signals in amemory card or the like.

Next, an example configuration of the color noise estimation section 14is described with reference to FIG. 2.

The color noise estimation section 14 comprises an OB region extractionsection 21 that extracts a signal of an OB (optical black) regionarranged on the right side of the image region of the CCD 4 as shown forexample in FIG. 4 from the image signals stored in the image buffer 8according to the control of the control section 18, a first buffer 22that stores the signals of the OB region extracted by the OB regionextraction section 21, a variance calculation section 23 constitutingvariance calculation means, that reads the signals of the OB regionstored in the first buffer 22 to calculate a variance value, then usesinformation such as that relating to the exposure conditions transferredfrom the control section 18 to correct the variance value with respectto the amplification value of the amplifier 6, a temperature estimationROM 25 constituting temperature estimation means in which thepre-measured relationship between the variance value and the temperatureof the image sensor is recorded, a temperature estimation section 24being a temperature estimation means and a parameter calculation meansfor determining the temperature of the CCD4, which is the image sensor,by referencing the temperature estimation ROM 25 based on the variancevalues outputted from the variance calculation section 23, a localregion extraction section 26 that extracts a local region of apredetermined size in a predetermined position from the image signalsstored in the image buffer 8, a second buffer 27 that stores the signalsof the local region extracted by the local region extraction section 26,an average luminance calculation section 28, which is a parametercalculation means, that reads out the signals of the local region storedin the second buffer 27 and calculates an average luminance value, again calculation section 29 being a gain calculation means, which isparameter calculation means for calculating an amplification value ofthe amplifier 6 based on information such as the exposure conditions,the ISO speed, and the white balance coefficient transferred from thecontrol section 18, a standard value applying section 30, which isapplying means for applying a standard value when any parameter isomitted, a color parameter ROM 32, which is coefficient calculationmeans, that stores color parameters related to a function, which will bedescribed later, used when estimating the color noise amount, acoefficient calculation section 31 being a coefficient calculation meansand a color noise amount calculation means that estimates the colornoise amount of the target pixel by means of the abovementionedfunction, on the base of information relating to the parameter read outfrom the color parameter ROM 32, the temperature of the image sensoroutputted from the temperature estimation section 24 or the standardvalue applying section 30, the average luminance value outputted fromthe average luminance calculation section 28 or the standard valueapplying section 30, and the amplification amount outputted from thegain calculation section 29 or the standard value applying section 30,and a function calculation section 33 being a function calculation meansand a color noise amount calculation means that uses the coefficientoutputted from the coefficient calculation section 31 to calculate thecolor noise amount using a function formulated as will be describedlater, and outputs to the color noise reducing section 13.

In the present embodiment, since processing by the color noise reducingsection 13, which is described later, is carried out in units of 4×4pixels, the local region extraction section 26 carries out extractionwhile progressively scanning the entire area of pixels using a 4×4 pixelunit. The processing by the local region extraction section 26 iscarried out synchronously with the processing of the color noisereducing section 13.

Furthermore, the control section 18 is bidirectionally connected to theOB region extraction section 21, the variance calculation section 23,the temperature estimation section 24, the local region extractionsection 26, the average luminance calculation section 28, the gaincalculation section 29, the standard value applying section 30, thecoefficient calculation section 31, and the function calculation section33, and is configured to control these.

A relationship between OB region variance and image sensor temperaturein the temperature estimation section 24 is described with reference toFIG. 5.

As is shown in this figure, the temperature of the image sensor rises ina monotonic increases while describing a curve according to increases inthe variance of the OB region.

In the case of random noise in the OB region, in which there is noincident light, dark current noise is governing factor, and this darkcurrent noise is related to the temperature of the image sensor.

For this reason, the random noise of the OB region is calculated as avariance value and the relationship between this variance value andtemperature variation of the image sensor is measured in advance andstored in the temperature estimation ROM 25. As a result, thetemperature estimation section 24 can estimate the temperature of theimage sensor CCD 4 from the variance values calculated by the variancecalculation section 23 using corresponding relationships stored in thetemperature estimation ROM 25.

On the other hand, the calculation of the average luminance value by theaverage luminance calculation section 28 is carried out as follows.

First, when the local region extraction section 26 carries outextraction of local regions in units of 4×4 pixels from the image signalof the CCD 4, which comprises a Bayer-type color filter as describedabove, block data is obtained arranged as shown in FIG. 3.

This block data of 4×4 pixels contains G signals of 8 pixels, R signalsof 4 pixels, and B signals of 4 pixels. Accordingly, hereinafter, Gpixels are given as G_(i) (i=1 to 8), R pixels are given as R_(j) (j=1to 4), and B pixels are given as B_(k) (k=1 to 4) and these are shownwith subscripts appended respectively. The pixel positions correspondingto these subscripts at this time are as shown in FIG. 3.

The average luminance calculation section 28 uses the G signal as asignal approximating the luminance signal and calculates the averageluminance as an average value G_(AV) as shown in the following formula1.

[Formula 1]

$G_{AV} = {\sum\limits_{i = 1}^{8}\frac{G_{i}}{8}}$

Next, the formulation of the color noise amount used when thecoefficient calculation section 31 estimates the color noise amount of atarget pixel is described with reference to FIG. 6A and FIG. 6B.

FIG. 6A plots color noise amounts N_(C(R−G)) and N_(C(B−G)) relating totwo types of color difference signals (R−G) and (B−G) corresponding toluminance levels. As shown in the diagram, the color noise amountsincrease linearly with respect to the luminance level.

When this variation in color noise amount is formulated using L as theluminance level and N_(C) as the color noise amount, the followingformula 2 is obtained.N _(C) =AL+B  [Formula 2]

Here, A and B are constant terms.

However, the color noise amount N_(C) is dependent not only on theluminance level L, but also varies due to such factors as thetemperature of the image sensor CCD 4 and the gain of the amplifier 6.Accordingly, an example giving consideration to these factors is shownin FIG. 6B.

FIG. 6B plots the color noise amount with respect to the luminancelevel, temperature, and gain. As shown in the diagram, each curve isformed as indicated by formula 2, but the coefficient varies dependingon the temperature and gain. Accordingly, when consideration is given tothese and formulation is carried out setting the temperature as T andthe gain as G, the following formula 3 is obtained.N _(C) =a(T, G)L+b(T, G)  [Formula 3]Here, a (T, G) and b (T, G) are functions that set the temperature T andthe gain G as parameters.

FIG. 7A shows an overview of the characteristics of the function a (T,G) and FIG. 7B shows an overview of the characteristics of the functionb (T, G).

Since these functions are two variable functions with the temperature Tand the gain G as independent variables, FIGS. 6A and 6B are plottedwith three dimensional coordinates and a curved surface in the plottedspace. However, instead of illustrating a specific curved surface formhere, the manner of variation in the characteristics is indicatedbroadly using a curved line.

The constant terms A and B are outputted by inputting the temperature Tand the gain G as parameters in the functions a and b. Then, a specificform of these functions can be easily obtained by measuring in advancethe characteristics of the image sensor including the CCD 4 and theamplifier 6.

The above-described two functions a (T, G) and b (T, G) are recordedseparately in the color parameter ROM 32 for each of the two types ofcolor difference signals (R−G) and (B−G).

The coefficient calculation section 31 calculates the coefficients A andB with the dynamically obtained (or obtained from the standard valueapplying section 30) temperature T and gain G as input parameters usingthe two functions a (T, G) and b (T, G) recorded in the color parameterROM 32.

The function calculation section 33 determines a function form forcalculating a color noise amounts N_(C(R−G)) and N_(C(B−G)) relating tothe two types of color difference signals (R−G) and (B−G) by applyingcoefficients A and B calculated by the coefficient calculation section31 in the above-described formula 3. Then, color noise amountsN_(C(R−G)) and N_(C(B−G)) relating to the two types of color differencesignals (R−G) and (B−G) are calculated using the signal value level Loutputted from the average luminance calculation section 28 via thecoefficient calculation section 31.

In this way, the luminance level L used in calculating the color noiseamount becomes an average value G_(AV) of the G signal calculated by theaverage luminance calculation section 28 as shown in formula 1.

It should be noted that it is not absolutely necessary for parameterssuch as the temperature T and the gain G to be obtained for eachshooting. For example, since the temperature T stabilizes after a fixedperiod of time has elapsed following the switching on the power supply,it is also possible that after this has become stable, the controlsection 18 can store the temperature information calculated by thetemperature estimation section 24 in the standard value applying section30, and temperature information read out from the standard valueapplying section 30 is used omitting subsequent calculation processing.Thus, the standard value applying section 30 is configured to set andoutput standard parameters when not obtaining parameters from thetemperature estimation section 24, the gain calculation section 29, orthe average luminance calculation section 28 and the control section 18as necessary, and therefore reliable processing can be carried out andit is possible to achieve increased speed and energy efficiency inprocessing. It should be noted that the standard value applying section30 can also output standard values for other necessary parameters.

The color noise amount calculated as described above by the functioncalculation section 33 is transferred to the color noise reducingsection 13.

Next, an example of a configuration of the color noise reducing section13 is described with reference to FIG. 8.

The color noise reducing section 13 comprises a local region extractionsection 41 that extracts a local region of a predetermined size in apredetermined position from the image buffer 8, a buffer 42 for storingimage data of the local region extracted by the local region extractionsection 41, a color difference calculation section 43, which iscalculation means for calculating a color difference from image datastored in the buffer 42, an average color difference calculation section44, which is calculation means for calculating an average colordifference based on color differences calculated by the color differencecalculation section 43, an allowable range setting section 45, which issetting means for setting an allowable range (a small amplitude value)relating to a color difference based on the average color differencecalculated by the average color difference calculation section 44 andthe color noise amount estimated by the color noise estimation section14, a color difference correction section 46, which is smoothing meansthat corrects the color difference outputted from the color differencecalculation section 43 based on the allowable range set by the allowablerange setting section 45, and an inverse transformation section 47,which is inverse transformation means for inversely transforming thecolor difference corrected by the color difference correction section 46to the original RGB signals or the like and outputting the inverselytransformed signals to the signal processing section 15.

Furthermore, the control section 18 is bidirectionally connected to thelocal region extraction section 41, the color difference calculationsection 43, the average color difference calculation section 44, theallowable range setting section 45, color difference correction section46, and the inverse transformation section 47, and is configured tocontrol these sections.

The local region extraction section 41 extracts image signals for eachpredetermined size from the image buffer 8, in units of 4×4 pixels inthe present embodiment for example, according to the control of thecontrol section 18, and transfers the extracted image signals to thebuffer 42.

The color difference calculation section 43 reads out the image signalsstored in the buffer 42 according to the control of the control section18 and calculates two types of color difference signals (R_(j)−G_(AV))and (B_(k)−G_(AV)) (j=1 to 4 and k=1 to 4). As shown in FIG. 3, foureach of R and B pixels are present in the 4×4 pixel local region, andtherefore four types each of the color difference signals are calculatedhere. It should be noted that G_(AV) refers to an average value of the Gsignals in the 4×4 pixels as indicated in formula 1.

The color difference signals calculated by the color differencecalculation section 43 are transferred to the average color differencecalculation section 44 and the color difference correction section 46respectively.

Based on the received two types of color difference signals(R_(j)−G_(AV)) and B_(k)−G_(AV)), the average color differencecalculation section 44 calculates average color difference valuesRG_(AV) and BG_(AV) respectively as shown in the following formula 4.

$\begin{matrix}{{{RG}_{AV} = {\sum\limits_{j = 1}^{4}\frac{R_{j} - G_{AV}}{4}}}{{BG}_{AV} = {\sum\limits_{k = 1}^{4}\frac{B_{k} - G_{AV}}{4}}}} & \left\lbrack {{Formula}\mspace{14mu} 4} \right\rbrack\end{matrix}$

Thus, the average color difference signals RG_(AV) and BG_(AV)calculated by the average color difference calculation section 44 aretransferred to the allowable range setting section 45.

Based on color noise amounts N_(C(R−G)) and N_(C(B−G)) relating to thetwo types of color difference signals (R_(j)−G_(AV)) and B_(k)−G_(AV))from the color noise estimation section 14 and the average colordifference values RG_(AV) and BG_(AV) from the average color differencecalculation section 44, the allowable range setting section 45 sets anupper limit U and a lower limit D as the allowable range (smallamplitude value) relating to the color noise amount as indicated in thefollowing formula 5.U _((R−G)) =RG _(AV) +N _(C(R−G))/2D _((R−G)) =RG _(AV) −N _(C(R−G))/2U _((B−G)) =BG _(AV) +N _(C(B−G))/2D _((B−G)) =BG _(AV) −N _(C(B−G))/2  [Formula 5]

Thus, the allowable range U and D calculated by the allowable rangesetting section 45 is transferred to the color difference correctionsection 46.

Based on the control of the control section 18, the color differencecorrection section 46 corrects (by absorbing amplitude components lowerthan the small amplitude value) the two types of color differencesignals (R_(j)−G_(AV)) and (B_(k)−G_(AV)) from the color differencecalculation section 43 according to the allowable range U and D from theallowable range setting section 45 and calculates color differencesignals (R_(j)−G_(AV))′ and (B_(k)−G_(AV))′ in which color noise isreduced.

At this time, the correction carried out by the color differencecorrection section 46 is divided into three types, namely, a case whereexceeding the upper limit U of the allowable range, a case where withinthe allowable range, and a case where below the lower limit D of theallowable range.

First, the color difference signal (R_(j)−G_(AV)) is shown in formula 6.When (R _(j) −G _(AV))>U _((R−G))(R _(j) −G _(AV))′=(R _(j) −G _(AV))−N _(C(R−G))/2When U _((R−G))≧(R _(j) −G _(AV))≧D _((R−G))(R _(j) −G _(AV))′=RG _(AV)When D _((R−G))>(R _(j) −G _(AV))(R _(j) −G _(AV))′=(R _(j) −G _(AV))+N_(C(R−G))/2  [Formula 6]

Next, the color difference signal (B_(k)−G_(AV)) is shown in formula 7.When (B _(k) −G _(AV))>U _((B−G))(B _(k) −G _(AV))′=(B _(k) −G _(AV))−N_(C(B−G))/2When U _((B−G))≧(B _(k) −G _(AV))≧D _((B−G))(B _(k) −G _(AV))′=BG _(AV)When D _((B−G))>(B _(k) −G _(AV))(B _(k) −G _(AV))′=(B _(k) −G _(AV))+N _(C(B−G))/2  [Formula 7]

All the two types of color difference signals (R_(j)−G_(AV)) and(B_(k)−G_(AV)) of which four each are present in the 4×4 pixels asdescribed above are corrected based on formula 6 or formula 7.

Thus, color difference signals that have undergone color correction bythe color difference correction section 46 are transferred to theinverse transformation section 47.

The inverse transformation section 47 transforms the color differencesignals to the original signals, which are RGB signals in the presentembodiment.

At this time, since it is desired that the G signals corresponding tothe luminance signals are maintained unchanged, correction of the colordifference signals is carried out on only the R signals or the Bsignals, such that a result of correction is transformation in which anR′ signal and a B′ signal are obtained. In the transformation here,three types (a total of six types) of calculations are carried outcorresponding to the three types (a total of six types) of each of thetwo types of color difference signals (R_(j)−G_(AV)) and (B_(k)−G_(AV)).

First, the color difference signal (R_(j)−G_(AV)) is shown in thefollowing formula 8.When (R _(j) −G _(AV))′=R _(j) −G _(AV) −N _(C(R−G))/2R _(j) ′=R _(j) −N _(C(R−G))/2When (R _(j) −G _(AV))′=RG _(AV)R _(j) ′=RG _(AV) +G _(AV)When (R _(j) −G _(AV))′=R _(j) −G _(AV) +N _(C(R−G))/2R _(j) ′=R _(j) +N _(C(R−G))/2  [Formula 8]

Next, the color difference signal (B_(k)−G_(AV)) is shown in formula 9.When (B _(k) −G _(AV))′=B _(k) −G _(AV) −N _(C(B−G))/2B _(k) ′=B _(k) −N _(C(B−G))/2When (B _(k) −G _(AV))′=BG _(AV)B _(k) ′=BG _(AV) +G _(AV)When (B _(k) −G _(AV))′=B _(k) −G _(AV) +N _(C(B−G))/2B _(k) ′=B _(k) +N _(C(B−G))/2  [Formula 9]

The formulas 8 and 9 refer to inverse transformation from the colordifference signals (R_(j)−G_(AV))′ and (B_(k)−G_(AV))′, in which colornoise has been reduced, to the original RGB signals. By carrying outthis inverse transformation, R′ signals and B′ signals having reducednoise and the original G signals are obtained. The thus-obtained RGBsignals (R′GB′ signals) are transferred to the signal processing section15.

Furthermore, the processing of the color noise reducing section 13 iscarried out synchronously with the calculation of color noise amount inthe color noise estimation section 14 according to the control section18.

It should be noted that in the above description, color noise amountsare estimated using a 4×4 pixel unit, but there is no limitation to thisconfiguration, and it is also possible to have a configuration in whichthe color noise amounts are estimated using very large regions as units,for example 8×8 pixels or 16×16 pixels. Although the accuracy of colornoise estimation is reduced when employing such a configuration, it hasthe advantage of enabling greater speed in processing.

Furthermore, in the above description, it is assumed that the processingis carried out using hardware, but there is no limitation to this andthe processing can be performed using software.

For example, the image signals from the CCD 4 are taken as raw data inan unprocessed state, and information from the control section 18 suchas the temperature and gain at the time of shooting and so on are addedto the raw data as header information. The raw data with appended headerinformation may be outputted to a processing device such as a computerand the processing may be performed by software in the processingdevice.

An example of color noise reduction processing using an image processingprogram in a computer is described with reference to FIG. 9.

When processing starts, first, all the color signals constituted of theraw data and the header information containing information abouttemperature and gain and so on is read (step S1).

Next, a local region of a predetermined size, for example a local regionusing 4×4 pixels as a unit, is extracted from the raw data (step S2).

Then, the signals of the extracted local regions are separated intocolor signals for each color filter and luminance signals and colordifference signals are calculated (step S3).

Following this, an average luminance value is calculated as shown informula 1 and an average color difference value is calculated as shownin formula 4 (step S4).

Further still, parameters such as temperature and gain are obtained fromthe header information that is read (step S5). If the necessaryparameters are not present in the header information here, apredetermined standard value is applied.

Next, a function such as that shown in formula 3 is read in (step S6)and the color noise amount is calculated (step S7) using the averageluminance value obtained in step S4 and the parameters such astemperature and gain obtained in step S5.

An upper limit and a lower limit are set (step S8) as the allowablerange as shown in formula 5 using the average color difference valueobtained in step S4 and the color noise amount obtained in step S7.

The processes of step S4 to step S8 are carried out as described aboveonly one time on a single local region.

Next, correction is carried out (step S9) as shown in formula 6 andformula 7 on the color difference signals obtained in step S3 based onthe allowable range obtained in step S8.

Inverse transformation is carried out (step S10) from the colordifference signals that are thus corrected to the original signals asshown in the formulas 8 and 9.

After this, a judgment is made (step S11) as to whether or notprocessing has finished concerning all the color difference signals inthe local region, and when this is not yet finished, the procedureproceeds to step S9 and processing is carried out as described above onthe next color difference signals.

On the other hand, when a judgment is made in step S11 that processingis finished concerning all the color difference signals, a furtherjudgment is made (step S12) as to whether or not processing has finishedon all the local regions.

Here, when processing has not finished concerning all the local regions,the procedure proceeds to step S2 and processing is carried outconcerning as described above with regard to the next local region.

Furthermore, when a judgment is made that processing is finishedconcerning all the local regions in step S12, commonly known enhancingprocess, compression process and the like are carried out (step S13).

Then, after the processed signals are outputted (step S14), the seriesof processes is finished.

With embodiment 1, color noise amounts are estimated using parametersthat vary dynamically such as the average luminance value and thetemperature and gain at the time of shooting, and therefore highlyaccurate color noise reduction processing can be carried out. And sincetemperature change detection is carried out using the OB region signal,an image pickup system can be achieved at low cost. Also, when theaforementioned parameters cannot be obtained, the color noise amount isestimated using a standard value, and therefore color noise amountreductions can be carried out reliably. Further still, by intentionallyomitting calculation of a portion of the parameters such as thetemperature of the image sensor after it has become stable, it ispossible to achieve an image pickup system having reduced costs andlower power consumption. Moreover, since the color noise reductionprocessing sets the allowable range from the color noise amount,reduction processing which is superior in terms of preservation of theoriginal signals can be accomplished. Additionally, since signals thathave undergone color noise reduction processing are inverselytransformed to the original signals, compatibility with conventionalprocessing systems is maintained and combinations involving varioussystems are possible. Furthermore, the luminance signals and the colordifference signals are obtained matching the arrangement of theBayer-type color filter, and therefore processing can be carried out athigh speed.

Embodiment 2

FIGS. 10 to 15 illustrate embodiment 2 of the present invention. FIG. 10is a block diagram illustrating a configuration of an image pickupsystem, FIG. 11 is a block diagram illustrating an example of aconfiguration of a luminance and color noise estimation section, FIG.12A and FIG. 12B illustrate an arrangement of a color-differenceline-sequential-type color filter, FIG. 13 is a block diagramillustrating a configuration of a luminance noise reducing section, FIG.14 is a block diagram illustrating another example of a configuration ofa luminance and color noise estimation section, and FIG. 15 is a flowchart illustrating a noise reducing process carried out by an imageprocessing program in a computer.

In the embodiment 2, the same sections are designated by the samereference numerals according to the embodiment 1, and are not described.Mainly, different points are described.

As shown in FIG. 10, in contrast to embodiment 1, the image pickupsystem of embodiment 2 additionally has a temperature sensor 51, whichis arranged near the CCD 4, measures the temperature of the CCD 4 inreal time, and outputs a measurement result to the control section 18,thereby constituting parameter calculation means; a luminance & colornoise estimation section 53, which is color noise estimation means andluminance noise estimation means for estimating not only the color noiseamount but also the luminance noise amount based on the image datastored in the image buffer 8; and a luminance noise reducing section 52,which is luminance noise reducing means, that reduces luminance noise ofimage data read out from the image buffer 8 based on the luminance noiseamount estimated by the luminance & color noise estimation section 53and outputs to the color noise reducing section 13, and deleted fromthis configuration is the color noise estimation section 14.

Furthermore, the color noise reducing section 13 receives estimatedcolor noise amounts from the luminance & color noise estimation section53 instead of the color noise estimation section 14. Further still, theoutput of the luminance noise reducing section 52 is configured to beinputted also to the luminance & color noise estimation section 53.

The control section 18 is bidirectionally connected to the luminancenoise reducing section 52 and the luminance & color noise estimationsection 53 and is configured such that it controls these sections.

Additionally, the present embodiment is described using an example inwhich the CCD 4 is a single CCD having a complementary color filterarranged in front, and this color filter has a color-differenceline-sequential-type arrangement as shown in FIG. 12A and FIG. 12B.

A configuration of a color-difference line-sequential-type color filteris described with reference to FIG. 12A and FIG. 12B.

The color-difference line-sequential-type color filter has 2×2 pixels asa basic unit in which one pixel each of cyan (C), magenta (M), yellow(Y), and green (G) are arranged. Note that the positions of the M and Gare inverted for each line.

The image signals from the CCD 4 provided with this color filter areseparated into an odd number field as shown in FIG. 12A and an evennumber field as shown in FIG. 12B, then outputted.

Furthermore, the signals saved to the image buffer 8 are not the CMYGsignals corresponding to the color filter constructed on the CCD 4, butrather a luminance signal L and color difference signals Cb and Cr,which are converted according to the following formula 10.L=C+M+Y+GCb=C+M−Y−GCr=M+Y−C−G  [Formula 10]

Accordingly, it is the luminance signals L and the color differencesignals Cb and Cr stored in the image buffer 8 that are transferred tothe luminance & color noise estimation section 53 and the luminancenoise reducing section 52.

Next, a flow of signals in the image pickup system shown in FIG. 10 thatis different from the image pickup system shown in FIG. 1 aresubstantially as follows.

Shooting conditions such as the white balance coefficient obtained bythe pre-white balance section 12, the exposure conditions obtained bythe exposure control section 9, the ISO speed set using the external I/Fsection 17, and the temperature of image sensor from the temperaturesensor 51 are transferred via the control section 18 to the luminance &color noise estimation section 53.

Based on this information, the luminance signals, and the colordifference signals, the luminance & color noise estimation section 53calculates the luminance noise amount and the color noise amount, aswill be described later, for each predetermined size, for example using4×4 pixels as a unit in the present embodiment.

The luminance noise amount calculated by the luminance & color noiseestimation section 53 is transferred to the luminance noise reducingsection 52 and the color noise amount is transferred to the color noisereducing section 13.

The calculations of luminance noise amount and color noise amount by theluminance & color noise estimation section 53 are carried outsynchronously with the processes of the luminance noise reducing section52 and the color noise reducing section 13 according to the controlsection 18.

Based on the luminance noise amounts transferred from the luminance &color noise estimation section 53, the luminance noise reducing section52 carries out luminance noise reduction processing on the luminancesignals read out from the image buffer 8 and transfers the processedimage signals to the luminance & color noise estimation section 53 andthe color noise reducing section 13.

The color noise reducing section 13 uses luminance signals in which theluminance noise has been reduced by the luminance noise reducing section52 and the original color difference signals to carry out color noisereduction processing on the color difference signals according to thecolor noise amount transferred from the luminance & color noiseestimation section 53, then transfers the processed image signals to thesignal processing section 15.

Based on the control of the control section 18, the signal processingsection 15 generates a single frame of image signals from even numberfield signals and odd number field signals of the luminance signals thathave undergone luminance noise reduction and the color differencesignals that have undergone color noise reduction, then carries outprocesses such as enhancing process, compression process and the like,and transfers to the output section 16.

Next, an example of a configuration of the luminance & color noiseestimation section 53 is described with reference to FIG. 11.

The luminance & color noise estimation section 53 has a processingsection for luminance noise estimation added to the color noiseestimation section 14 shown in FIG. 2 of embodiment 1 and a processingsection for temperature estimation is omitted thereof. Then theluminance & color noise estimation section 53 is configured to operatesuch that it first carries out estimation of the luminance noise amount,outputs the estimation result to the luminance noise reducing section52, then estimates the next color noise amount using the luminancesignal in which luminance noise has been reduced by the luminance noisereducing section 52 and outputs the estimation result to the color noisereducing section 13.

That is, the luminance & color noise estimation section 53 comprises thelocal region extraction section 26 that extracts local regions of apredetermined size in a predetermined position synchronously with theprocessing of the luminance noise reducing section 52 based on the imagesignals stored in the image buffer 8 or the corrected luminance signalsfrom the luminance noise reducing section 52, a buffer 61 for storingsignals of the local regions extracted by the local region extractionsection 26, a signal separation section 62, which is separation meansfor separating the luminance signals L and the color difference signalsCb and Cr from the signals of the local region stored in the buffer 61,an average calculation section 63, which is calculation means forcalculating an average luminance value from the luminance signalsseparated by the signal separation section 62, the gain calculationsection 29 that calculates the amplification amount of the amplifier 6according to the exposure conditions transferred from the controlsection 18 and the white balance coefficient and the like, the standardvalue applying section 30, which is applying means for applying astandard value when any of the parameters is omitted, the colorparameter ROM 32, which is coefficient calculation means for storing thecolor parameters relating to the function used in estimating the colornoise amount, a luminance parameter ROM 64, which is coefficientcalculation means for storing the luminance parameters relating to afunction that is described later and which is used to estimate theluminance noise amount, the coefficient calculation section 31, whichcombines color noise amount calculation means, luminance noise amountcalculation means, and coefficient calculation means that estimate thecolor noise amount and luminance noise amount of a target pixel using apredetermined formula based on information such as the parameters readout from the color parameter ROM 32 and the luminance parameter ROM 64,the amplification amount that is outputted from the gain calculationsection 29 or the standard value applying section 30, the temperature ofthe image sensor outputted from the control section 18 or the standardvalue applying section 30, and the average luminance value outputtedfrom the average calculation section 63, and the function calculationsection 33, which combines color noise amount calculation means,luminance noise amount calculation means, and function calculation meansthat calculate the color noise amount and the luminance noise amountusing functions that are formulated in a manner to be described based onthe coefficients outputted from the coefficient calculation section 31and output to the color noise reducing section 13 and the luminancenoise reducing section 52.

Furthermore, the control section 18 is bidirectionally connected to thelocal region extraction section 26, the signal separation section 62,the average calculation section 63, the gain calculation section 29, thestandard value applying section 30, the coefficient calculation section31, and the function calculation section 33, and is configured tocontrol these sections.

The local region extraction section 26 extracts signals of apredetermined size and predetermined position from the image buffer 8and transfers these to the buffer 61. In the present embodiment, aprocess of the luminance noise reducing section 52 that is to bedescribed later is carried out using 4×4 pixels as a unit, and thereforethe local region extraction section 26 extracts in units of 4×4 pixelswhile progressively scanning the entire surface of the image. Theprocessing of the local region extraction section 26 is carried outunder the control of the control section 18 in synchronization with theprocessing of the luminance noise reducing section 52.

As described above, the signals stored in the image buffer 8 are dividedinto an even number field and an odd number field. Accordingly,hereinafter description is given using as an example the odd numberfield shown in FIG. 12A, but the same is true for the even number field.

Based on the control of the control section 18, the signal separationsection 62 separates the luminance signals L_(i) (i=1 to 6) and thecolor difference signals Cb_(j) and Cr_(k) (j=1 to 3, k=1 to 3), whichare stored in the buffer 61.

That is, the luminance signals L_(i) and the color difference signalsCr_(k) that are obtained in the first line of the 4×4 pixels stored inthe buffer 61 as shown in FIG. 12A are calculated as shown in formula 11below.L ₁ =C ₁ +M ₁ +Y ₁ +G ₁L ₂ =C ₂ +M ₁ +Y ₁ +G ₂L ₃ =C ₂ +M ₂ +Y ₂ +G ₂Cr ₁ =M ₁ +Y ₁ −C ₁ −G ₁Cr ₂ =M ₁ +Y ₁ −C ₂ −G ₂Cr ₃ =M ₂ +Y ₂ −C ₂ −G ₂  [Formula 11]

Furthermore, the luminance signals L_(i) and the color differencesignals Cb_(j) that are obtained in the third line of the 4×4 pixelsstored in the buffer 61 are calculated as shown in formula 12 below.L ₄ =C ₃ +M ₃ +Y ₃ +G ₃L ₅ =C ₄ +M ₄ +Y ₃ +G ₃L ₆ =C ₄ +M ₄ +Y ₄ +G ₄Cb ₁ =C ₃ +M ₃ −Y ₃ −G ₃Cb ₂ =C ₄ +M ₄ −Y ₃ −G ₃Cb ₃ =C ₄ +M ₄ −Y ₄ −G ₄  [Formula 12]

Based on the control of the control section 18, the average calculationsection 63 reads out the luminance signals L_(i) from the buffer 61 andcalculates an average luminance value L_(AV) as shown in the followingformula 13 and transfers this to the coefficient calculation section 31.

$\begin{matrix}{L_{AV} = {\sum\limits_{i = 1}^{6}\frac{L_{i}}{6}}} & \left\lbrack {{Formula}\mspace{11mu} 13} \right\rbrack\end{matrix}$

On the other hand, the gain calculation section 29 calculates theamplification amount in the amplifier 6 based on information transferredfrom the control section 18 such as the exposure conditions, the ISOspeed, and the white balance coefficient, and transfers a calculationresult to the coefficient calculation section 31.

Based on the average luminance value L_(AV) from the average calculationsection 63, information about gain from the gain calculation section 29,and temperature of the image sensor measured by the temperature sensor51 and transferred from the control section 18, the coefficientcalculation section 31 estimates the luminance noise amount according toa formulation as shown below in formula 14 and formula 15.

That is, when the luminance level is set as L and the luminance noiseamount is set as N_(L), then:N _(L) =AL ^(B)+Γ  [Formula 14]OrN _(L) =AL ² +BL+Γ  [Formula 15]

It should be noted that A, B, and Γ in the formulas 14 and 15 areconstant terms.

Hereinafter description is given of an example using a formulationaccording to formula 14, but the same is true when using a formulationaccording to formula 15.

However, the luminance noise amount N_(L) is not dependent on only thesignal value level L, but also varies for other factors such as thetemperature of the image sensor CCD 4 and the gain of the amplifier 6.Accordingly, the following formula 16 is obtained when formulation iscarried out giving consideration to these factors.N _(L)=α(T, G)L ^(β(T, G))+γ(T, G)  [Formula 16]

Here, the temperature is set as T and the gain is set as G, and α (T,G), β (T, G), and γ (T, G) are functions acting as parameters for thetemperature T and the gain G. The specific forms of these functions caneasily be acquired by measuring, in advance, the characteristics of theimage pickup system including the CCD 4 and the amplifier 6.

The three above-described functions α (T, G), β (T, G), and γ (T, G) arestored in the luminance parameter ROM 64.

The coefficient calculation section 31 calculates the coefficients A, B,and Γ using the three functions α (T, G), β (T, G), and γ (T, G) storedin the luminance parameter ROM 64 with the temperature T and the gain Gas input parameters.

By applying the coefficients A, B, and Γ calculated by the coefficientcalculation section 31 to formula 16 (or formula 14), the functioncalculation section 33 determines a form of the function for calculatingthe luminance noise amount N_(L). Then, the luminance noise amount N_(L)is calculated using the signal level L (that is, the average signallevel L_(AV) shown in formula 13), which is outputted from the averagecalculation section 63 via the coefficient calculation section 31.

It should be noted that as is the same in the case of obtaining thecolor noise amount as described in embodiment 1, it is not absolutelynecessary to obtain parameters such as the temperature T and the gain Gevery shooting.

The luminance noise amount N_(L) that is calculated by the functioncalculation section 33 is transferred to the luminance noise reducingsection 52. As is described below, the luminance noise reducing section52 calculates a luminance signal L′_(i) in which luminance noise hasbeen reduced based on the luminance noise amount N_(L) that has beentransferred.

Next, based on the control of the control section 18, the local regionextraction section 26 transfers the luminance signals L′_(i) from theluminance noise reducing section 52 to the buffer 61.

At this time, the color difference signals, from the color noisereducing section 13, in which color noise has been reduced are not sentto the buffer 61, and therefore present in the buffer 61 are theluminance signals L′_(i) whose luminance noise has been reduced by theluminance noise reducing section 52 and the original color differencesignals Cb_(j) and Cr_(k).

Based on the control of the control section 18, the signal separationsection 62 separates and reads out the luminance signals L′_(i) and thecolor difference signals Cb_(j) and Cr_(k) that are stored in the buffer61.

Based on the control of the control section 18, the average calculationsection 63 reads out the luminance signals L′_(i) from the buffer 61,calculates an average luminance value L′_(AV) as shown in the followingformula 17, and transfers this to the coefficient calculation section31.

$\begin{matrix}{L_{AV}^{\prime} = {\sum\limits_{i = 1}^{6}\frac{L_{i}^{\prime}}{6}}} & \left\lbrack {{Formula}\mspace{11mu} 17} \right\rbrack\end{matrix}$

Based on the average luminance value L′_(AV) transferred from theaverage calculation section 63, the gain, and the temperature of theimage sensor, the coefficient calculation section 31 estimates colornoise amounts N_(C(Cb)′) and N_(C(Cr)). Here, the information of thegain and the temperature of the image sensor is the same as that usedwhen estimating the luminance noise amount as described above, andtherefore these may be stored in the standard value applying section 30and read out from the standard value applying section 30 when estimatingthe color noise amount.

Furthermore, the two functions a(T, G) and b(T, G) necessary forestimating the color noise amount are recorded separately in the colorparameter ROM 32 every two color difference signals Cb and Cr in thesame manner as embodiment 1.

The coefficient calculation section 31 calculates the coefficients A andB with the temperature T and the gain G as input parameters using thetwo functions a(T, G) and b(T, G) recorded in the color parameter ROM32.

By applying the coefficients A and B calculated by the coefficientcalculation section 31 to formula 3 (or formula 2), the functioncalculation section 33 determines the form of the functions forcalculating the color noise amount N_(C(Cb)′) and N_(C(Cr)) relating tothe two types of color difference signals Cb and Cr. Then, the colornoise amounts N_(C(Cb)′) and N_(C(Cr)) relating to the color differencesignals Cb and Cr are calculated using the signal value level L (thatis, the average luminance value L′_(AV) as shown in formula 17)outputted from the average calculation section 63 via the coefficientcalculation section 31.

Thus, the color noise amount calculated by the function calculationsection 33 is transferred to the color noise reducing section 13.

Next, an example configuration of the luminance noise reducing section52 is described with reference to FIG. 13.

The luminance noise reducing section 52 comprises a local regionextraction section 71 that extracts a local region of a predeterminedsize from the image buffer 8, a buffer 72 for storing the image data ofthe local region extracted by the local region extraction section 71, asignal separation section 73 serving as means for separating luminancesignals and color difference signals from the image signals stored inthe buffer 72, an average calculation section 74 that calculates anaverage luminance value based on the luminance signals separated by thesignal separation section 73, an allowable range setting section 75,which is setting means for setting an allowable range (a small amplitudevalue) relating to luminance based on the average luminance valuecalculated by the average calculation section 74 and the luminance noiseamount estimated by the luminance & color noise estimation section 53,and a signal correction section 76, which is smoothing means, thatcorrects the luminance signals outputted by the signal separationsection 73 based on the allowable range set by the allowable rangesetting section 75 and outputs these to the color noise reducing section13 and the luminance & color noise estimation section 53.

Furthermore, the control section 18 is bidirectionally connected to thelocal region extraction section 71, the signal separation section 73,the average calculation section 74, the allowable range setting section75, and the signal correction section 76, and is configured to controlthese sections.

Based on the control of the control section 18, the local regionextraction section 71 extracts image signals from the image buffer 8 foreach predetermined size, for example, for each 4×4 pixels in the presentembodiment, and transfers the extracted image signals to the buffer 72.

As described above, the image signals stored in the image buffer 8 aredivided into an even number field and an odd number field, and althoughdescription is given hereinafter using as an example the odd numberfield shown in FIG. 12A, the same is true for the even number field.

Based on the control of the control section 18, the signal separationsection 73 separates the luminance signals L_(i) (i=1 to 6) and thecolor difference signals Cb_(j) and Cr_(k) (j=1 to 3, k=1 to 3) from theimage signals stored in the buffer 72. Then, the separated luminancesignals L_(i) therein are transferred to the average calculation section74 and the signal correction section 76 respectively. It should be notedthat the color difference signals Cb_(j) and Cr_(k) are sent as they areto the color noise reducing section 13 via the signal correction section76.

The average calculation section 74 calculates the average luminancevalue L_(AV) based on the aforementioned formula 13 and the calculatedaverage luminance value L_(AV) is transferred to the allowable rangesetting section 75.

Based on the control of the control section 18, the allowable rangesetting section 75 sets an upper limit U and a lower limit D as theallowable range (small amplitude value) relating to the luminance noiseamount based on the luminance noise amount N_(L) relating to theluminance signals from the luminance & color noise estimation section 53and the average luminance value L_(AV) from the average calculationsection 74 as indicated in the following formula 18.U _(L) =L _(AV) +N _(L)/2, D _(L) =L _(AV) −N _(L)/2  [Formula 18]

Thus, the allowable range U and D calculated by the allowable rangesetting section 75 is transferred to the signal correction section 76.

Based on the control of the control section 18, the signal correctionsection 76 corrects (by absorbing amplitude components lower than thesmall amplitude value) the luminance signals L_(i) from the signalseparation section 73 according to the allowable range U and D from theallowable range setting section 75 and calculates luminance signalsL′_(i) in which luminance noise is reduced.

At this time, the correction carried out by the signal correctionsection 76 is divided into three types, namely, a case where exceedingthe upper limit U of the allowable range, a case where within theallowable range, and a case where below the lower limit D of theallowable range, and is carried out as shown in the following formula19.When L_(i)>U_(L) , L′ _(i) =L _(i) −N _(L)/2When U_(L)≧L_(i)≧D_(L), L′_(i)=L_(AV)When D_(L)>L_(i) , L′ _(i) =L _(i) +N _(L)/2  [Formula 19]

There are six types of the luminance signals L_(i) in the 4×4 pixels,and correction based on formula 19 is carried out on all these luminancesignals.

Thus, the luminance signals L_(i) that have undergone correction by thesignal correction section 76 and the original color difference signalsCb_(j) and Cr_(k) are transferred to the color noise reducing section13.

The color noise reducing section 13 carries out color noise reductionbased on the color noise amounts N_(C(Cr)) and N_(C(Cb)) in the samemanner as embodiment 1.

Namely, first, an average color difference value Cb_(AV) relating to thecolor difference signal Cb_(j) and an average color difference valueCr_(AV) relating to the color difference signal Cr_(k) are calculated asindicated in the following formula 20.

$\begin{matrix}{{{Cb}_{AV} = {\sum\limits_{j = 1}^{3}\frac{{Cb}_{j}}{3}}}{{Cr}_{AV} = {\sum\limits_{k = 1}^{3}\frac{{Cr}_{k}}{3}}}} & \left\lbrack {{Formula}\mspace{11mu} 20} \right\rbrack\end{matrix}$

Next, based on the thus-calculated average color difference valuesCb_(AV) and Cr_(AV) and the color noise amounts N_(C(Cb)) and N_(C(Cr))transferred from the luminance & color noise estimation section 53, theupper limit U and the lower limit D are set as indicated by thefollowing formula 21 as the allowable range for color noise amounts.U _((Cb)) =Cb _(AV) +N _(C(Cb))/2D _((Cb)) =Cb _(AV) −N _(C(Cb))/2U _((Cr)) =Cr _(AV) +N _(C(Cr))/2D _((Cr)) =Cr _(AV) −N _(C(Cr))/2  [Formula 21]

Then, correction is carried out on the color difference signals Cb_(j)and Cr_(k) based on these allowable ranges and color difference signalsCb_(j)′ and Cr_(k)′ in which color noise has been reduced are calculatedrespectively by the following formula 22 and formula 23.When Cb_(j)>U_((Cb)) , Cb _(j) ′=Cb _(j) −N _(C(Cb))/2When U_((Cb))≧Cb_(j)≧D_((Cb)), Cb_(j)′=Cb_(AV)When D_((Cb))>Cb_(j) , Cb _(j) ′=Cb _(j) +N _(C(Cb))/2  [Formula 22]When Cr_(k)>U_((Cr)) , Cr _(k) ′=Cr _(k) −N _(C(Cr))/2When U_((Cr))≧Cr_(k)≧D_((Cr)), Cr_(k)′=Cr_(AV)When D_((Cr))>Cr_(k) , Cr _(k) ′=Cr _(k) +N _(C(Cr))/2  [Formula 23]

In this way, the luminance signals L′_(i) in which luminance noise hasbeen reduced by the luminance noise reducing section 52 and the colordifference signals Cb_(j)′ and Cr_(k)′ in which color noise has beenreduced by the color noise reducing section 13 are transferred to thesignal processing section 15.

It should be noted that description is given above using an example of acomplementary color-based color-difference line-sequential-type singleCCD, but of course there is no limitation to this and, for example, aprimary color Bayer-type single CCD may be used as is described inembodiment 1, and additionally it is possible a two CCD or a three CCD.For example, when using the primary color Bayer-type image sensor ofthese, the G signal may be used as the luminance signal and (R−G_(AV))and (B−G_(AV)) may be used as the color difference signals.

Furthermore, the aforementioned luminance & color noise estimationsection 53 carries out calculations using functions to estimate theluminance noise amounts and the color noise amounts, but there is nolimitation to this, and a configuration using a look-up table forexample is also possible.

A configuration example of the luminance & color noise estimationsection 53 is described with reference to FIG. 14.

The basic construction of the luminance & color noise estimation section53 shown in FIG. 14 is similar to that of the luminance & color noiseestimation section 53 shown in FIG. 11, however, this embodiment differsfrom the FIG. 11 in that a luminance noise table 81 and a color noisetable 82 is provided instead of the coefficient calculation section 31,the color parameter ROM 32, the function calculation section 33, and theluminance parameter ROM 64.

That is to say, the luminance noise table 81 constituting look-up tablemeans and luminance noise amount calculation means, and the color noisetable 82 constituting look-up table means and color noise amountcalculation means input information from the abovementioned averagecalculation section 63, gain calculation section 29 and standard valueapplying section 30.

Furthermore, the output of the luminance noise table 81 is inputted tothe luminance noise reducing section 52 and the output of the colornoise table 82 is inputted to the color noise reducing section 13.

Also, the control section 18 is bidirectionally connected to theluminance noise table 81 and the color noise table 82, and is configuredto control these tables.

Next, areas of the function of the luminance & color noise estimationsection 53 shown in FIG. 14 that are different than the function of theluminance & color noise estimation section 53 shown in FIG. 11 aremainly as follows.

Based on the control of the control section 18, the average calculationsection 63 calculates the average luminance value L_(AV) as shown informula 13 and first transfers this to the luminance noise table 81.

Similarly, gain information from the gain calculation section 29 andinformation relating to the temperature of the image sensor from thecontrol section 18 are respectively transferred to the luminance noisetable 81.

The luminance noise table 81 is a table that records luminance noiseamounts for the parameters of luminance level, temperature, and gain,which are calculated in advance based on formulation of the luminancenoise as shown in formula 16.

Consequently, it is possible to directly obtain luminance noise amountsfrom the luminance noise table 81 according to the transferred data.

Thus, luminance noise amounts obtained by referencing the luminancenoise table 81 are transferred to the luminance noise reducing section52.

When processing by the luminance noise reducing section 52 is carriedout using the transferred luminance noise amounts, the processingresults are transferred to the local region extraction section 26 whereas described above processing is carried out.

As a result, based on the control of the control section 18, the averagecalculation section 63 calculates the average luminance value L′_(AV)after luminance noise reduction has been carried out as shown in formula17, then transfers this to the color noise table 82.

Similarly, gain information from the gain calculation section 29 (or thestandard value applying section 30) and information relating to thetemperature of the image sensor from the control section 18 (or thestandard value applying section 30) are transferred to the color noisetable 82 respectively.

The color noise table 82 is a table that records color noise amounts forthe parameters of luminance level, temperature, and gain, which arecalculated in advance based on formulation of the color noise as shownin formula 3.

Consequently, it is possible to directly obtain color noise amounts fromthe color noise table 82 according to the transferred data.

Thus, color noise amounts obtained by referencing the color noise table82 are transferred to the color noise reducing section 13.

With a configuration using the luminance noise table 81 and the colornoise table 82, the computational processing for calculating theluminance noise and color noise can be omitted and therefore processingcan be achieved at higher speeds.

Furthermore, the above-described processes, which are assumed to beexecuted using hardware, can also be performed using software in aprocessing device such as a computer in the same manner as embodiment 1.

An example of carrying out noise reduction processing using an imageprocessing program on a computer is described with reference to FIG. 15.

When processing starts, first, all the color signals constituted of theraw data and the header information containing information abouttemperature and gain and so on is read (step S21).

Next, a local region of a predetermined size, for example a local regionusing 4×4 pixels as a unit, is extracted from the raw data (step S22).

Then, the signals of the extracted local regions are separated intoluminance signals and color difference signals (step S23).

Following this, an average luminance value is calculated as shown informula 13 (step S24).

Further still, parameters such as temperature and gain are obtained fromthe header information that is read (step S25). If the necessaryparameters are not present in the header information here, apredetermined standard value is applied.

Next, a table relating to luminance noise amounts is read in and theluminance noise amount is obtained using the average luminance valueobtained in step S24 and the parameters such as temperature and gainobtained in step S25 (step S26).

The processes in the above-described step S23 to step S26 are carriedout one time only on a single local region.

An upper limit and a lower limit are set as the allowable range bycarrying out the calculation shown in formula 18 based on the luminancenoise amount obtained in step S26, and correction is carried out asshown in formula 19 on the luminance signals obtained in step S23 (stepS27).

Then, a judgment is made as to whether or not processing has finishedconcerning all the luminance signals in the local region (step S28), andwhen this is not yet finished, the procedure proceeds to step S27 andprocessing is carried out as described above on the next luminancesignals.

On the other hand, when a judgment is made in step S28 that processingis finished concerning all the luminance signals, then the luminancesignals whose luminance noise has been corrected and the original colordifference signals are separated (step S29).

After this, the average luminance value is calculated as shown informula 17 and the average color difference value is calculated as shownin formula 20 (step S30).

Further still, parameters such as temperature and gain are obtained fromthe header information that has been read in (step S31). If thenecessary parameters are not present in the header information here, apredetermined standard value is applied. in the same manner as describedabove.

Next, the table relating to color noise amounts is read and the colornoise amount is obtained using the average luminance value obtained instep S30 and parameters such as temperature and gain obtained in stepS31 (step S32).

The upper limit and lower limit are set as the allowable range bycarrying out calculation as shown in formula 21 based on the averagecolor difference value obtained in step S30 and the color noise amountobtained in step S32, and correction is carried out as shown in formula22 and formula 23 on the color difference signals obtained in step S29(step S33).

Then, a judgment is made as to whether or not processing has finishedconcerning all the color difference signals in the local region (stepS34), and when this is not yet finished, the procedure proceeds to stepS33 and processing is carried out as described above on the next colordifference signals.

On the other hand, when a judgment is made in step S34 that processingis finished concerning all the color difference signals, a judgment ismade as to whether or not processing has finished on all the localregions (step S35). Here, when it is judged that processing has notfinished, the procedure proceeds to step S22 and processing is carriedout concerning as described above with regard to the next local region.

Furthermore, when a judgment is made in step S35 that processing isfinished concerning all the local regions, commonly known enhancingprocess, compression process and the like are carried out (step S36).

Then, after the processed signals are outputted (step S37), the seriesof processes is finished.

With embodiment 2, luminance noise amounts and color noise amounts areestimated using parameters that vary dynamically such as the averageluminance value and the temperature and gain at the time of shooting,and therefore highly accurate luminance noise reduction processing andcolor noise reduction processing can be carried out. And sincetemperature change detection is carried out using a dedicated sensor,noise amounts can be estimated with high accuracy. Also, when theaforementioned parameters cannot be obtained, the luminance noise amountand the color noise amount are estimated using standard values, andtherefore luminance noise reductions and color noise amount reductionscan be carried out reliably. Further still, by intentionally omittingcalculation of a portion of the parameters such as the temperature ofthe image sensor after it has become stable, it is possible to achievean image pickup system having reduced costs and lower power consumption.Moreover, since the luminance noise reduction processing sets theallowable range from the luminance noise amount, reduction processingwhich is superior in terms of preservation of the original signals canbe accomplished. Additionally, since the color noise reductionprocessing sets the allowable range from the color noise amount,reduction processing which is superior in terms of preservation of theoriginal signals can be accomplished. Furthermore, the luminance signalsand the color difference signals are obtained matching the arrangementof the color-difference line-sequential-type color filter, and thereforeprocessing can be carried out at high speed.

It should be noted that the present invention is not limited to theabove-described embodiments and various modifications and applicationswill of course become possible without departing from the scope thereof.

1. An image pickup system for noise reduction processing on signals froman image sensor, in front of which is arranged a color filter, saidsystem comprising: calculation means for calculating luminance signalsand color difference signals from the signals in each of predeterminedunit regions, calculating an average luminance value based on thecalculated luminance signals for each predetermined unit region, andcalculating an average color difference value based on the calculatedcolor difference signals for each predetermined unit region, color noiseestimation means for estimating a color noise amount related to thecolor difference signals for each predetermined unit region based on theaverage luminescence value, and color noise reducing means for reducingcolor noise in the color difference signals based on the color noiseamount and the average color difference value.
 2. The image pickupsystem according to claim 1, wherein the color filter is one of aprimary color filter and a complementary color filter.
 3. The imagepickup system according to claim 1, wherein the color filter is aBayer-type primary color filter constituted of R (red), G (green), and B(blue), wherein the calculation means calculates a G pixel average valueG_(AV) as the average luminance value, calculates as the colordifference signals a color difference signal R−G_(AV) for each R pixelin the predetermined unit region and a color difference signal B−G_(AV)for each B pixel in the predetermined unit region, and calculates anaverage color difference value RG_(AV) based on the calculated colordifference signal R−G_(AV) and an average color difference value BG_(AV)based on the calculated color difference signal B−G_(AV) as the averagecolor difference values, wherein the color noise estimation meansestimates a color noise amount related to the color difference signalR−G_(AV) and the color difference signal B−G_(AV) based on the averageluminance value G_(AV,) and wherein the color noise reducing meansreduces color noise in the color difference signal R−G_(AV) and colornoise in the color difference signal B−G_(AV) respectively based on thecolor noise amount and on the average color difference value RG_(AV) andthe average color difference value BG_(AV).
 4. The image pickup systemaccording to claim 1, wherein the color filter is a color-differenceline-sequential-type complementary color filter of C (cyan), M(magenta), Y (yellow), and G (green), wherein the calculation meanscalculates a luminance signal L=C+M−Y−G as the luminance signal,calculates a color difference signal Cb=C+M−Y−G and a color differencesignal Cr=M+Y−C−G as the color difference signal, calculates an averageluminance value L_(AV) as the average luminance value, and calculates anaverage color difference value Cb_(AV) and an average color differencevalue Cr_(AV) as the average color difference values, and wherein thecolor noise estimation means estimates a color noise amount related tothe color difference signals Cb and Cr based on the average luminancevalue L_(AV), and wherein the color noise reducing means reduces colornoise in the color difference signal Cb and the color difference signalCr respectively based on the color noise amount and on the average colordifference values Cb_(AV) and Cr_(AV).
 5. The image pickup systemaccording to claim 1, wherein the predetermined unit region is a regionof a unit of 4×4 pixels.
 6. The image pickup system according to claim1, further comprising: separation means for separating from the signalsthe luminance signal or the color signal for each color filter,luminance noise estimation means for estimating a luminance noise amountin the color signals or the luminance signal for each predetermined unitregion, and luminance noise reducing means for reducing luminance noisein the color signals or the luminance signals based on the luminancenoise amount for each predetermined unit region, wherein the calculationmeans calculates luminance signals and color difference signals for eachof predetermined unit regions based on signals in which luminance noisehas been reduced by the luminance noise reducing means.
 7. The imagepickup system according to claim 1, wherein the color noise estimationmeans comprises: parameter calculation means for obtaining a parameterbased on information of at least one of the average luminance value, atemperature of the image sensor, and a gain with respect to the signals,and color noise amount calculation means for obtaining a color noiseamount based on the parameter.
 8. The image pickup system according toclaim 6, wherein the luminance noise estimation means comprises:parameter calculation means for obtaining a parameter based oninformation of at least one of an average value of the color signals, anaverage value of the luminance signals, a temperature of the imagesensor, and a gain with respect to the signals, and luminance noiseamount calculation means for obtaining a luminance noise amount based onthe parameter.
 9. The image pickup system according to claim 7, whereinthe parameter calculation means comprises a temperature sensor formeasuring a temperature of the image sensor.
 10. The image pickup systemaccording to claim 8, wherein the parameter calculation means comprisesa temperature sensor for measuring a temperature of the image sensor.11. The image pickup system according to claim 7, wherein, the imagesensor comprises an OB (optical black) region, and the parametercalculation means comprises variance calculation means for determiningvariance of signals in the OB (optical black) region in the imagesensor, and with temperature estimation means for estimating atemperature of the image sensor based on the variance.
 12. The imagepickup system according to claim 8, wherein, the image sensor comprisesan OB (optical black) region, and the parameter calculation meanscomprises variance calculation means for determining variance of signalsin the OB (optical black) region in the image sensor, and withtemperature estimation means for estimating a temperature of the imagesensor based on the variance.
 13. The image pickup system according toclaim 7, wherein the parameter calculation means comprises gaincalculation means for determining the gain based on information of atleast one of an ISO speed, exposure information, and white balanceinformation.
 14. The image pickup system according to claim 8, whereinthe parameter calculation means comprises gain calculation means fordetermining the gain based on information of at least one of an ISOspeed, exposure information, and white balance information.
 15. Theimage pickup system according to claim 7, wherein the parametercalculation means is configured to calculate as parameters an averageluminance value L, a temperature T of the image sensor, and a gain G ofthe signals, wherein the color noise estimation means further comprisesapplying means for applying a standard parameter value in regard toparameters unobtainable from the parameter calculation means, andwherein the color noise amount calculation means calculates a colornoise amount N_(C) using a parameter obtained from the parametercalculation means or the applying means, and comprises: (i) coefficientcalculation means for calculating two coefficients A and B based on twofunctions a (T, G) and b (T, G) using the temperature T and the gain Gas parameters; and (ii) function calculation means for calculating thecolor noise amount N_(C) based on a functional equation N_(C)=AL+Bdefined by the two coefficients A and B calculated by the coefficientcalculation means.
 16. The image pickup system according to claim 7,wherein the color noise estimation means further comprises applyingmeans for applying a standard parameter value in regard to parametersunobtainable from the parameter calculation means, and wherein the colornoise amount calculation means comprises look-up table means forobtaining a color noise amount based on an average luminance value, atemperature of the image sensor, and a gain of the signals, which areobtained from the parameter calculation means or the applying means. 17.The image pickup system according to claim 8, wherein the parametercalculation means is configured to calculate as parameters an averagevalue L of the color signals or the luminance signals, a temperature Tof the image sensor and a gain G of the signals, wherein the luminancenoise estimation means further comprises applying means for applying astandard parameter value in regard to parameters unobtainable from theparameter calculation means, and wherein the luminance noise amountcalculation means calculates a luminance noise amount N_(L) using aparameter obtained from the parameter calculation means or the applyingmeans, and comprises: (i) coefficient calculation means for calculatingthree coefficients A, B, and Γ based on three functions α(T, G), β(T,G), and γ(T, G) using the temperature T and the gain G parameters; and(ii) function calculation means for calculating the luminance noiseamount N_(L) based on either a first functional equation N_(L)=AL^(B)+Γor a second functional equation N_(L)=AL²+BL+Γ defined by the threecoefficients A, B, and Γ calculated by the coefficient calculationmeans.
 18. The image pickup system according to claim 8, wherein theluminance noise estimation means further comprises applying means forapplying a standard parameter value in regard to parameters unobtainablefrom the parameter calculation means, and the luminance noise amountcalculation means comprises look-up table means for obtaining aluminance noise amount based on an average value of the color signals oran average value of the luminance signals, a temperature of the imagesensor, and a gain of the signals, which are obtained from the parametercalculation means or the applying means.
 19. The image pickup systemaccording to claim 1, wherein the color noise reducing means comprises:setting means for setting a small amplitude value for each predeterminedunit region based on a color noise amount from the color noiseestimation means, and smoothing means for absorbing amplitude componentsless than the small amplitude value with respect to the color differencesignals.
 20. The image pickup system according to claim 6, wherein theluminance noise reducing means comprises: setting means for setting asmall amplitude value for each predetermined unit region based on aluminance noise amount from the luminance noise estimation means, andsmoothing means for absorbing amplitude components less than the smallamplitude value with respect to the color signals or the luminancesignal.
 21. The image pickup system according to claim 1, furthercomprising inverse transformation means for inversely transforming colordifference signals, in which color noise has been reduced by the colornoise reducing means, to signals of a same type as signals from theimage sensor.
 22. An image pickup method for controlling an image pickupsystem to execute functions comprising: calculating luminance signalsand color difference signals in each of predetermined unit regions fromsignals from an image sensor, in front of which is arranged a colorfilter, calculating an average luminance value based on the calculatedluminance signals for each predetermined unit region, and calculating anaverage color difference value based on the calculated color differencesignals for each predetermined unit region, estimating a color noiseamount related to the color difference signals for each predeterminedunit region based on the average luminescence value, and reducing colornoise in the color difference signals based on the color noise amountand the average color difference value.
 23. The image pickup methodaccording to claim 22, wherein the image pickup method further controlsthe image pickup system to execute functions comprising: separatingluminance signals or color signals in each color filter from the signalsfrom the image sensor, estimating a luminance noise amount in the colorsignals or the luminance signals for each predetermined unit region, anda reducing luminance noise in the color signals or the luminance signalsbased on the luminance noise amount for each predetermined unit region,wherein the luminance signals and color difference signals for eachpredetermined unit region are calculated based on signals in whichluminance noise has been reduced.
 24. The image pickup method accordingto claim 22, wherein a parameter is obtained based on information of atleast one of the average luminance value, a temperature of the imagesensor, and a gain with respect to the signals, and a color noise amountis obtained based on the parameter.
 25. The image pickup methodaccording to claim 24, wherein an average luminance value L, atemperature T of the image sensor, and a gain G of the signals arecalculated, wherein a standard parameter value is applied with regard toparameters unobtainable from the parameter calculation procedure,wherein a color noise amount N_(C) is calculated using an obtainedparameter wherein two coefficients A and B are calculated based on twofunctions a(T, G) and b(T, G) using the temperature T and the gain G asparameters; and wherein the color noise amount N_(C) is calculated basedon a functional equation N_(C)=AL+B defined by the two coefficients Aand B calculated.
 26. The image pickup method according to claim 22,wherein a small amplitude value for each predetermined unit region isset based on a color noise amount, and amplitude components less thanthe small amplitude value are absorbed with respect to the colorsignals.